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(54) Method and apparatus for three-dimensional image rendering of body organs 



(57) A method for mapping a structure in a body of 
a subject includes capturing a three-dimensional (3D) 
imfige of the structure comprising diagnostic informa- 
tion, and generating a 3D geometrical map of the struc- 
ture using a probe inserted into the structure. The image 



FIG. 1 



is registered with the map, such that each of a plurality 
of image points in the image is identified with a corre- 
sponding map point in the map. The map is displayed 
such that the diagnostic information associated with 
each of the image points is displayed at the correspond- 
ing map point. 
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Description 

FIELD OF THE INVENTION 

5 [0001] The present invention relates generally to systems and methods for three-dimensional mapping and recon- 
struction, and specifically to mapping and reconstruction of the interior of body organs, such as the heart. 

BACKGROUND OF THE INVENTION 

10 [0002] Various methods of diagnostic imaging are known in the art. Methods used for imaging the heart, for example, 
include fluoroscopy, angiography, echocardiography, computed tomography (CT), magnetic resonance imaging (MRI), 
positron emission tomography (PET) and single photon emission tomography (SPECT). Many of these methods pro- 
duce three-dimensional (3D) image information, which can then be rendered for viewing in the form of parallel slices 
through the heart,orasapseudo-3D display on a video monitor. In order to administer treatment, the treating physician 

is must build a 3D picture in his or her mind based on the two-dimensional pictures that are displayed. The transposition 
is particularly tricky when therapy is to be administered inside the heart, such as local electrical ablation of aberrant 
electrical pathways, or laser myocardial revascularization. 

[0003] It is also known in the art to map the heart using a mapping probe, typically a catheter, inside the heart 
chambers. Exemplary methods and devices for this purpose are described in U.S. Patents 5,471,982 and 5,391 ,199 

20 and in PCT patent publications WO94/06349, WO96/05768 and W097/24981, whose disclosures are incorporated 
herein by reference. U.S. Patent 5,391 ,1 99, for example, describes a catheter that includes both electrodes for sensing 
cardiac electrical activity and miniature coils for determining the position of the catheter relative to an externally-applied 
magnetic field. Using this catheter a cardiologist can collect data from a set of sampled points in the heart within a 
short period of time, by measuring the electrical activity at a plurality of locations and determining the spatial coordinates 

25 of the locations. Locations of the mapping catheter within the heart can be superimposed on a 3D reconstruction of an 
image of the heart, such as an ultrasound image, acquired prior to or during the catheter study. Color codes are used 
to represent electrical activity sensed by the catheter. 

[0004] U.S. Patent 5,738,096, whose disclosure is incorporated herein by reference, describes methods for geomet- 
rical mapping of the endocardium based on bringing a probe into contact with multiple locations on a wall of the heart, 

30 and determining position coordinates of the probe at each of the locations. The position coordinates are combined to 
form a map of at least a portion of the heart. Once the position of the catheter is known, external sensors can be used 
to provide local physiological values of heart tissue adjacent to the tip of the catheter. For example, if the catheter 
incorporates a radioactive marker suitable for SPECT, local functional information can be gleaned from a SPECT image. 
Yet another example is determining local perfusion from Doppler-ultrasound images of the coronaries, from nuclear 

35 medicine images or from X-ray or CT angiography, and overlaying the perfusion map on the geometrical map. The 
image of the catheter in the perfusion map can be used to align the perfusion map and the geometrical map. Alterna- 
tively, the alignment may be carried out using fiducial marks or anatomical reference locations, either automatically or 
manually. 

[0005] Further methods for creating a three-dimensional map of the heart based on these data are disclosed, for 
40 example, in European patent application EP 0 974 936 and in EP-A-0 974 936 whose disclosure is incorporated herein 
by reference. As indicated in these applications, position coordinates (and optionally electrical activity, as well) are 
initially measured at about 1 0 to 20 points on the interior surface of the heart. These data points are generally sufficient 
to generate a preliminary reconstruction or map of the cardiac surface to a satisfactory quality. The preliminary map 
is preferably combined with data taken at additional points in order to generate a more comprehensive map. 

45 

SUMMARY OF THE INVENTION 

[0006] It is an object of some aspects of the present invention to provide improved methods and apparatus for map- 
ping and visualization of internal body structures, and particularly of the heart. 
50 [0007] It is a further object of some aspects of the present invention to provide improved methods and apparatus for 
administering local treatment of pathological conditions within the heart. 

[0008] in preferred embodiments of the present invention, a position-sensing catheter is used to generate a 3D 
geometrical map of the internal surface of a heart chamber of a subject. A 3D diagnostic image of the heart is captured 
in conjunction with generating the 3D map, typically either before or concurrently with the mapping. The image and 
55 map are brought into mutual registration, and diagnostic information from the image, such as perfusion information, is 
then marked on the 3D map, preferably in the form of color coding. Based on the combined diagnostic and geometrical 
information, a physician operating the catheter is able to identify and visualize areas of the heart that are in need of 
treatment, due to low perfusion, for example. The physician preferably uses the catheter to apply a local invasive 
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therapy, such as laser revascularization, to specific points that are located using the color-coded 3D map. Alternatively, 
a local diagnostic technique, such as a biopsy, may be performed at such specific points. 

[0009] There is therefore provided, in accordance with a preferred embodiment of the present invention, a method 
for mapping a structure in a body of a subject, including: 

5 

capturing a three-dimensional (3D) image of the structure including diagnostic information; 
generating a 3D geometrical map of the structure using a probe inserted into the structure; 
registering the image with the map, such that each of a plurality of image points in the image is identified with a 
corresponding map point in the map; and 
10 displaying the map, such that the diagnostic information associated with each of the image points is displayed at 
the corresponding map point. 

y. . [0010] In a preferred embodiment, the diagnostic information is related to blood flow in the structure, wherein the 
diagnostic information includes local perfusion data. In other preferred embodiments, the diagnostic information In- 

is eludes metabolic data, or is related to uptake of a substance in tissue of the structure, or is related to motion of the 
structure. 

[0011] Preferably, generating the geometrical map includes bringing the. probe into contact with the structure at a 
multiplicity of locations on the structure, and recording position coordinates of the probe at the locations, wherein 
recording the position coordinates includes determining the coordinates using a position sensor in the probe. 

20 [001 2] Preferably, registering the image with the map includes applying a transformation to at least one of the image 
and the map so that following the transformation, the image and the map have a common axis and a common scale. 
Further preferably, registering the image with the map includes dividing the Image into a plurality of parallel planar 
slices, perpendicular to the axis and mutually spaced along the axis, wherein the plurality of image points are located 
in the slices. More preferably, registering the image with the map includes finding an axial coordinate of each of the 

25 slices and an angular coordinate of each of the image points located in each of the slices, and identifying each of the 
image points with the map point having the same axial and angular coordinates. Most preferably, the structure includes 
a wall defining a cavity, and identifying each of the image points with the map point includes finding, at the axial and 
the angular coordinate, the image point that is within a section of the wall. 

[0013] Preferably, displaying the map includes coloring the map to reflect the diagnostic information. 

30 [001 4] In a preferred embodiment, the method includes performing a medical procedure on the structure guided by 
the diagnostic information displayed on the map. Preferably, performing the medical procedure includes using the 
probe to perform the procedure locally at locations selected on the geometrical map, and the method includes marking 
on the geometrical map the locations at which the procedure was performed. Additionally or alternatively, performing 
the medical procedure includes performing a therapeutic procedure, wherein the diagnostic information relates to local 

35 blood flow in the structure, and wherein performing the therapeutic procedure includes performing a procedure for 
improving the local blood flow. Alternatively, performing the medical procedure includes performing a diagnostic pro- 
cedure. 

[001 5] Preferably, the structure includes a heart of the subject, and generating the geometrical map includes mapping 
an endocardial surface in a ventricle of the heart. 
40 [0016] There is also provided, in accordance with a preferred embodiment of the present invention, apparatus for 
mapping a structure in a body of a subject, including: 

an imaging device, adapted to capture a three-dimensional (3D) image of the structure including diagnostic infor- 
mation; 

45 a probe, adapted to be inserted into the structure, so as to generate a 3D geometrical map of the structure; 

a processor, coupled to the probe and to the Imaging device, and adapted to register the image with the map, such 
that each of a plurality of image points in the image is identified with a corresponding map point in the map; and 
a display, coupled to be driven by the processor to d isplay the map, such that the diagnostic information associated 
with each of the image points is displayed at the corresponding map point. 

so 

[0017] The present invention will be more fully understood from the following detailed description of the preferred 
embodiments thereof, taken together with the drawings in which; 

BRIEF DESCRIPTION OF THE DRAWINGS 

55 

[0018] 

Fig. 1 is a schematic, pictorial illustration of a system for imaging, mapping and treatment of the heart, in accordance 
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with a preferred embodiment of the present invention; 

Fig. 2 is a flow chart that schematically illustrates a method for imaging, mapping and treating the heart, in accord- 
ance with a preferred embodiment of the present invention; 

s 

Fig. 3 is a schematic representation of a map of a chamber of the heart, in accordance with a preferred embodiment 
of the present invention; 

Fig. 4 is a simplified geometrical representation of the map of Fig. 3, showing coordinates used in registering the 
io map with an image of the heart, in accordance with a preferred embodiment of the present invention; 

Fig. 5 is a schematic, exploded view of a 3D image of the heart, represented as a stack of parallel slices through 
the heart, in accordance with a preferred embodiment of the present invention; 

is Fig. 6 shows the slices of Fig. 5 arrayed side-by-side, illustrating registration of the slices with the 3D map of Fig. 

3, in accordance with a preferred embodiment of the present invention; and 

Fig. 7 is a schematic representation of the map of Fig. 3, after coloring of the map with diagnostic information from 
the image of Figs. 5 and 6, in accordance with a preferred embodiment of the present invention. 

20 

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS 

[001 9J Fig, 1 is a schematic, pictorial illustration of a system 20 for three-dimensional geometrical mapping, imaging 
and treatment of a heart 24 of a subject 26 , in accordance with a preferred embodiment of the present invention. System 
25 20 comprises an elongate probe, preferably a catheter 30, which Is inserted by a user 22 through a vein or artery of 
the subject into a chamber of the heart. 

[0020] Catheter 30 preferably comprises at least one position sensor (not shown in the figures), most preferably 
located near the catheter's distal tip. The position sensor preferably comprises an electromagnetic sensor, which is 
mounted within Ihe catheter by any suitable method, for example, using polyurethane glue or the like. The sensor is 

30 electrically connected to an electromagnetic sensor cable, which extends through the catheter body and into a control 
handle of the catheter. In the control handle, the wires of the sensor cable are connected to a circuit board (not shown), 
which amplifies the signals received from the electromagnetic sensor and transmits them to a computer housed in a 
console 34, in a form understandable to the computer Because the catheter is designed for single use only, the circuit 
board preferably contains an EPROM chip, which shuts down the circuit board after the catheter has been used. This 

35 prevents the catheter, or at least the electromagnetic sensor, from being used twice. 

[0021 ] To use the electromagnetic sensor, subject 26 is placed in a magnetic field generated, for example, by situating 
under the patient a pad containing field generator coils 28 for generating a magnetic field, driven by driver circuits 32. 
A reference electromagnetic sensor (not shown) is preferably fixed relative to the patient, e.g., taped to the patient's 
back, and catheter 30 containing its sensor is advanced into heart 24. The sensor preferably comprises three small 

40 coils, which in the magnetic field generate weak electrical signals indicative of their position in the magnetic field. 
Signals generated by both the fixed reference sensor and by the sensor in the heart are amplified and transmitted to 
console 34, which analyzes the signals and then displays theresults on a monitor 36. By this method, the precise 
location of the sensor in the catheter relative to the reference sensor can be ascertained and visually displayed. The 
sensors can also detect displacement of the catheter that is caused by contraction of the heart muscle. 

45 [0022] Suitable electromagnetic sensors for the purposes of the present Invention are described, for example, In the 
above-mentioned U.S. Patent 5,391 ,1 99 and PCT patent publication WO 96/05768. A preferred electromagnetic map- 
ping sensor is manufactured by Biosense Ltd. (Tirat Hacarmel, Israel) and marketed under the trade designation NOGA. 
Some of the mapping features of catheter 30 and system 20 are implemented in the NOGA-STAR catheter marketed 
by Biosense Webster, Inc., and in the Biosense-NOGA system, also marketed by Biosense Webster, Inc. Further 

so aspects of the design of catheter 30 and of system 20 generally are described in EP-A-1 125 549 which is incorporated 
herein by reference. Using such sensors, system 20 achieves continuous generation of six dimensions of position and 
orientation information with respect to catheter 30. Alternatively, the sensors used in catheter 20 may comprise other 
types of position and/or coordinate sensors, as described, for example, in U.S. Patent 5,391,199, 5,443,489 or 
5,515,853, or in PCT publication WO 94/04938 or WO 99/05971 , or substantially any other suitable type of position/ 

55 coordinate sensing device known in the art. 

[0023] As noted above, catheter 30 is coupled to console 34, which enables the user to observe and regulate the 
functions of the catheter. Console 34 includes a processor, preferably a computer with appropriate signal processing 
circuits (which are typically contained inside a housing of the computer). The processor is coupled to drive display 36. 
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User 22 brings the distal tip of catheter 30 into contact with multiple points on the endocardial surface of heart 24, and 
the position coordinates are recorded at each point. The information derived from this analysis is used to reconstruct 
a three-dimensional geometrical map 38 of the endocardial surface of heart 24. 

[0024] System 20 also comprises a diagnostic imaging unit 48, such as an echo Doppler unit, SPECT, PET, MRI, 

5 CT or other imaging unit known in the art. Unit 48 is used to capture a 3D diagnostic image of heart 24, preferably 
while user 22 is mapping the heart using catheter 30. Alternatively, the diagnostic image is captured before beginning 
the mapping, and unit 48 may, in this case, be separate from the other elements of system 20. Diagnostic data from 
the image captured by unit 48 are superimposed on map 38, using methods described hereinbelow. Depending on the . 
type and configuration of unit 48, a wide range of different diagnostic data may be represented in the image, such as 

w perfusion, metabolic factors, uptake of markers, heart wall motion or thickness, and/or other anatomical or electrical 
properties, as are known in the art. The image can also be timed to represent different phases In the heart cycle. 
[0025] Typically, system 20 includes other elements, some of which are not shown in the figures for the sake of 
a simplicity. In the present embodiment, the system preferably includes a laser console 49, which is used in performing 
direct myocardial revascularization, as described, for example, in PCT patent application PCT/IL97/00011 and in U.S. 

15 patent application 09/1 09,820, which is assigned to the assignee of the present patent application and whose disclosure 
is incorporated herein by reference. Console 49 injects laser energy into a suitable waveguide (not shown) within 
catheter 30. The waveguide conveys the energy to the distal tip of the catheter, where It is applied to revascularlze 
areas of the myocardium suffering from low perfusion. Alternatively, the system may Include other therapeutic elements, 
as are known in the art, particularly elements for delivering local treatment in the heart, such as a radio-frequency 

20 driver coupled to an ablation electrode on catheter 30; an ultrasound generator coupled to high-power transducer in 
the catheter, for ultrasonic ablation of the endocardium; or a supply of a therapeutic agent, such as growth factors for 
angiogenesis, coupled to an injection needle in the catheter. Still further alternatively, the system may include invasive 
diagnostic elements, such as biopsy forceps that are operated through catheter 30. 

[0026] Other elements that may be comprised in system 20 are described, for example, in EP-A-0 974 936. 

25 [0027] Typically, system 20 includes an ECG monitor (not shown), coupled to receive signals from one or more body 
surface electrodes, so as to provide an ECG synchronization signal to console 34. As mentioned above, the system 
preferably also includes a reference position sensor, either on an externally-applied reference patch attached to the 
exterior of the patient's body, or on an internally-placed catheter, which is inserted into heart 24 and maintained In a 
fixed position relative to the heart. By comparing the position of catheter 30 to that of the reference catheter, the 

30 coordinates of catheter 30 are accurately determined relative to the heart, irrespective of heart motion. Alternatively, 
any other suitable method may be used to compensate for heart motion. 

[0028] Fig. 2 is a flow chart that schematically illustrates a method for imaging, mapping and treatment of heart 24 
using system 20, in accordance with a preferred embodiment of the present invention. At an imaging step 50, a diag- 
nostic image of heart 24, such as a SPECT image, is captured. Preferably, although not necessarily, the image is 

35 captured while catheter 30 is already located inside the heart. The catheter is used to generate geometrical map 38, 
at a mapping step 52. Suitable mapping techniques forthis purpose are described In the above-mentioned U.S. Patent 
5,738,096. The above-mentioned European patent application EP 0 974 936 describe accurate methods for creating 
the map itself based on the data gathered using catheter 30. The image captured at step 50 and the map created at 
step 52 are then registered one with the other, at a registration step 54. 

40 [0029] Figs. 3 and 4 are schematic representations of map 38 generated by system 20 at step 52, illustrating a 
method used at registration step 54, in accordance with a preferred embodiment of the present invention. Fig. 3 is a 
wire frame rendition of the map, representing the left ventricle of heart 24. For the purposes of step 54, a longitudinal 
axis 72 is drawn through the map, passing through an apex 74 of the ventricle. Preferably, the axis and apex are found 
automatically by console 34. Alternatively or additionally, these or other features of the map are identified manually by 

« user 22. 

[0030] Fig. 4 is a simplified geometrical representation of a surface 80 of map 38, generated for the purpose of 
registration with a diagnostic image of heart 24. Surface 80 corresponds to an approximate locus of the endocardium 
of the heart, as determined from map 38. A coordinate system is defined in which each point 82 on surface 80 is 
represented by a distance R from apex 74 and an angle a relative to a downward direction 84 (i.e., the direction pointing 

so toward the feet of subject 26). 

[0031] In order to register the diagnostic image with map 38, axis 72 and apex 74 are Identified in the image, as well, 
and are aligned with the axis and apex of the map. The identification is preferably automatic but may, alternatively or 
additionally, be carried out or assisted by user 22. Other landmarks and/or fiducial marks in the heart can also be used 
in performing the alignment. The scale of the image is adjusted so that its dimensions match those of the map as 

55 closely as possible. For many types of diagnostic images, such as perfusion maps, the resolution of the diagnostic 
information is low, so that imprecision of as much as 10 mm in mutual registration can be tolerated. When higher 
resolution is required, the registration of the diagnostic Image with the geometrical map may be improved using methods 
of automatic registration such as those described in Appendix A. These methods are optional and are not essential to 
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* 

the present invention. 

[0032] Rg. 5 is a schematic, exploded view of a 3D diagnostic image 90 of heart 24, following registration of the 3D 
image with geometrical map 38, in accordance with a preferred embodiment of the present Invention. This view is 
generated at a bullseye rendition step 56 In the method of Fig, 2. The bullseye rendition of Image 90 comprises a stack 
5 of parallel slices 92, which are perpendicular to axis 72. The slices are preferably taken at a fixed slice increment one 
from another along the axis. Each slice shows a section 94 of image 90, at a distance R from apex 74 that is determined 
by the slice number. 

[0033] Fig. 6 shows slices 92 of image 90 arrayed side-by-side, illustrating extraction of diagnostic data from the 
slices for application to map 38, in accordance with a preferred embodiment of the present invention. Referring, for 
io example, to slice number 5, sectional image 94 comprises three essential parts: an inner region 100, showing the 
inside of the ventricle; a wall region 102, showing the myocardium; and an outer region 104, external to the heart. The 
diagnostic information of interest is in region 1 02. Assuming image 90 to be a SPECT image, showing perfusion in the 
heart wall, for example, region 102 will typically have the highest value of perfusion. 

[0034] At a coloration transfer step 58, the diagnostic information from each slice 92 is transferred to map 38. Each 

15 slice has a known value of distance R from apex 74. For each angle a within the slice, point 82 on surface 80 of the 
map (Fig. 4) is assumed to be the point at that angle that Is located radially In the middle of region 102. In the case 
that Image 90 Is a perfusion image, point 82 Is simply taken to be the point of highest perfusion at the given angle. In 
other Imaging modalities, finding region 1 02 Is, for the most part, similarly straightforward. The value of the diagnostic 
data at each point 82 is preferably represented as a color applied to the corresponding region of map 38, 

20 [0035] Fig. 7 is a schematic representation of a colored geometrical map 11 0, as produced at step 58, in accordance 
with a preferred embodiment of the present invention. Because of the limited ability of a line drawing to convey qualities 
of a color image, only two different color regions appear on map 110: a well-perfused region 112, and an ischemic 
region 114. Preferably, the ischemic region has a darker or "cooler*' color than the well-perfused region. In actual 
applications, in which display 36 comprises a color monitor, a broad range of different colors is used in map 110 to 

25 describe different levels of perfusion or of other diagnostic qualities. 

[0036] Preferably, system 20 is operated by user 22 to carry out an invasive therapeutic procedure, guided by map 
110, at a therapeutic step 60. In the present example, laser console 49 is operated to irradiate ischemic region 114 via 
catheter 30 with high-intensity laser radiation, as described in the above-mentioned PCT patent application PCT/ 
IL97/00011. The laser creates revascularization channels in the myocardium, which are marked by system 20 with 

30 spots 1 16 on map 110. The combination of the imaging, mapping and therapeutic modalities enables the user to con- 
centrate the treatment in the region of heart 24 that is known to need it, and to ensure that the region is fully covered. 
Other local therapeutic and diagnostic procedures can similarly benefit from the guidance provided by map 110. 
[0037] Although preferred embodiments are described hereinabove with reference to heart 24, the principles of the 
present invention may similarly be applied to imaging, mapping and treatment of other organs and body structures. It 

35 will thus be appreciated that the preferred embodiments described above are cited by way of example, and that the 
present invention is not limited to what has been particularly shown and described hereinabove. Rather, the scope of 
the present invention includes both combinations and subcombinations of the various features described hereinabove, 
as well as variations and modifications thereof which would occur to persons skilled in the art upon reading the foregoing 
description and which are not disclosed in the prior art. 

40 

APPENDIX A 

r 

[0038] This appendix provides details of step 54 in the method of Fig. 2, in which two 3D representations, Pand O, 
of a chamber of heart 24 are brought into mutual registration. To begin, a rough estimate is found for the transformation 
45 between Pand O, either manually, or using the principle axis of a bounding ellipsoid or by principle component de- 
composition. The bounding ellipsoid technique Is further described In the above-mentioned European patent application 
EP 0 974 936 

[0039] Fine registration between P and Q is then preferably found using a variation of the Iterative Closest Point 
(ICP) algorithm. This algorithm is described by Besl and McKay in "A Method for Registration of 3D Shapes," published 
so in IEEE Transactions on Pattern Analysis and Machine Intelligence 1 4(2):239-256 (1 992), which is incorporated herein 
by reference. The following steps are repeated until convergence: 

1. Nearest point search: For each point p in P find the closest point q on Q. One can take a subset of points of P 
to improve computation speed. Similarly, all points of Q can be covered, too, to ensure robustness. 
55 2. Compute registration: Evaluate a transformation T that minimizes the sum of squared distances between pairs 

of closest points (p,q). The transformation is preferably either rigid, similarity, aff ine or projective as described below. 
3. Transform: Apply the transformation T to all points in P. 
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[0040] Given two surfaces, P and O, and two sets of points, {p, G P} .feeO) , step 2 of this algorithm seeks 
a transformation, 7, from a family of transformations (according to the possible f amiliefta escribed below) that minim izes 
the mean square error, e, between the corresponding sets: 



i = l 



10 



Affine and Projective Transformations 

[O04:hK,.For affine transformations, defined as l\p) + Ap+ t, 
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is a translation vector. 
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we have three systems of equations: 
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[0042] Let a singular value decomposition of X be X = UDtA. It then follows that a y = VDlflyj 
[0043] Projective transformations are evaluated in a similar way to the affine case. 
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Similarity and Rigid Transformation 



[0044] Whereas in affine transforms triangles are transformed to triangles, similarity transformations preserve pro- 
portions. We seek a scaling factor, c, a 3x3 rotation matrix, R, and a 3-dimensional translation vector, U such that T 
(p) = cRp + 1, wherein the error 



10 



<?(R, Ts c) « - £ bi * ten + 4 

1=1 



15 



is minimized. 

[0045] A suitable method for finding the desired similarity transform is described by Umeyama, in "Least-Squares 
Estimation of Transformation Parameters Between Two Point Patterns," published in IEEE Transactions on Pattern 
Analysis and Machine Intelligence, 13(4): 376-380 (1 991 ), which is incorporated herein by reference. Define the center 
of mass of both P and Q. 
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[0046] Then define the variance of the points on both P and O,: 
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[0047] The covariance matrix between the two surfaces is 



45 



Zpg = £ fei " *j) (Pi - Ap) 

i = 1 



so [0048] Let a singular value decomposition of £ p(7 be E w - t/0V T , and 
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S = < 



I if det(C7) det(v) = 1 

diag(lA-l) if det(rj)det(v) = -1 
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[0049] The rotation, translation and scaling of the transformation are then given by: 

R = USV T 

c = tracefos) 

wherein the trace of a matrix is the sum of its diagonal elements. 
[0050] In the case of rigid transformation no scaling is applied, so that c = 1 . 

75 Claims 

1 . A method for mapping a structure in a body of a subject, comprising: 

capturing a three-dimensional (3D) image of the structure comprising diagnostic information; 
20 generating a 3D geometrical map of the structure using a probe inserted into the structure; 

registering the image with the map, such that each of a plurality of image points in the image is identified with 
a corresponding map point in the map; and 

displaying the map, such that the diagnostic information associated with each of the image.points is displayed 
at the corresponding map point. 

25 

2. A method according to claim 1 , wherein the diagnostic information: 

is related to blood flow in the structure; 
comprises local perfusion data; 
30 comprises metabolic data; 

is related to uptake of a substance in tissue of the structure; or 
is related to motion of the structure. 

3. The method of claim 1 or claim 2 wherein generating the geometrical map comprises bringing the probe into contact 
35 with the structure at a multiplicity of locations on the structure, and recording position coordinates of the probe at 

the locations. 

4. The method of claim 3, wherein recording the position coordinates comprises determining the coordinates using 
a position sensor in the probe. 

40 

5. The method of any one of claims 1 to 4, wherein registering the image with the map comprises applying a trans- 
formation to at least one of the image and the map so that, following the transformation, the image and the map 
have a common axis and a common scale. 

45 6. The method of claim 5, wherein registering the image with the map comprises dividing the image into a plurality 
of parallel planar slices, perpendicular to the axis and mutually spaced along the axis, and wherein the plurality of 
image points are located in the slices. 

7. The method of claim 6, wherein registering the image with the map comprises finding an axial coordinate of each 
so of the slices and an angular coordinate of each of the image points located in each of the slices, and identifying 

each of the image points with the map point having the same axial and angular coordinates. 

6. The method of claim 7, wherein the structure comprises a wall defining a cavity, and wherein identifying each of 
the image points with the map point comprises finding, at the axial and the angular coordinate, the image point 

55 that is within a section of the wall. 

9. The method of any one of claims 1 to 8, wherein displaying the map comprises coloring the map to reflect the 
diagnostic information. 
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The method of any one of claims 1 to 9, wherein the structure comprises a heart of the subject, and wherein 
generating the geometrical map comprises mapping an endocardial surface in a ventricie of the heart. 

Apparatus for mapping a structure in a body of a subject, comprising: 

an imaging device, adapted to capture a three-dimensional (3D) image of the structure comprising diagnostic 
information; 

a probe, adapted to be inserted Into the structure, so as to generate a 3D geometrical map of the structure; 
a processor, coupled to the probe and the imaging device, and adapted to register the image with the map, 
such that each of a plurality of image points in the image Is identified with a corresponding map point in the 
map; and 

a display, coupled to be driven by the processor to display the map, such that the diagnostic information 
associated with each of the image points is displayed at the corresponding map point. 

*5 1 2. Apparatus of claim 1 1 , wherein the diagnostic information: 

Is related to blood flow In the structure; 
comprises local perfusion data; 
comprises metabolic data; 
20 is related to uptake of a substance in tissue of the structure; or 

is related to motion of the structure. 

13. Apparatus of claim 11 or claim 12, wherein, to generate the geometrical map, the probe is adapted to be brought 
into contact with the structure at a multiplicity of locations on the structure, and the processor is adapted to record 

» position coordinates of the probe at the locations. 

14. Apparatus of claim 13, wherein the probe comprises a position sensor for use in determining the position coordi- 
nates. 

jo 1 5. Apparatus of any one of claims 1 1 to 1 3, wherein the processor is adapted to register the image with the map by 
applying a transformation to at least one of the image and the map so that following the transformation, the image 
and the map have a common axis and a common scale. 

1 6. Apparatus of claim 1 5 , wherein the processor is further adapted to divide the image Into a plurality of parallel planar 
J* slices, perpendicular to the axis and mutually spaced along the axis, wherein the plurality of image points are 

located in the slices. 

17. Apparatus of claim 16, wherein the processor is adapted to find an axial coordinate of each of the slices and an 
angular coordinate of each of the image points located in each of the slices, and to identify each of the image 

-to points with the map point having the same axial and angular coordinates. 

18. Apparatus of claim 17, wherein, when the structure comprises a wall defining a cavity, the processor is adapted 
to identify each of the image points with the map point by finding, at the axial and the angular coordinate, the image 
point that is within a section of the wall. 

45 

19. Apparatus of any one of claims 11 to 1B, which is adapted to provide a map colored to reflect the diagnostic 
information. 

20. Apparatus of any one of claims 11 to 19, wherein the structure comprises a heart of the subject, and wherein the 
so geometrical map comprises a map of an endocardial surface in a ventricle of the heart. 

21. Apparatus of any one of claims 11 to 20, and comprising a medical instrument adapted to perform a medical 
procedure on the structure guided by the diagnostic information displayed on the map. 

55 22. Apparatus of claim 21, wherein the medical instrument is contained in the probe, which is adapted to be used to 
perform the procedure locally at locations selected on the geometrical map. 

23. Apparatus of claim 22, wherein the processor is adapted to mark on the geometrical map the locations at which 
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the procedure was performed. 

24. Apparatus of any one of claims 21 to 23 for carrying out therapeutic procedure. 

5 25. Apparatus of claim 24, wherein the diagnostic Information relates to local blood flow in the structure, for carrying 
out a therapeutic procedure for improving the local blood flow. 

26. Apparatus of any one of claims 21 to 23 for carrying out a diagnostic procedure. 

10 



15 



20 



25 



30 



35 



40 



45 



50 



55 



11 



BNSOOCtO: <EP. 



11B261BA2J_> 



EP 1 182 619 A2 




BNSDOCID: <EP. 



1tB2610A2J_> 



EP 1 182 619 A2 



FIG. 2 



CAPTURE DIAGNOSTIC IMAGE 



SO 

rv 



GENERATE 3-D MAP 



I 



52 



REGISTER IMAGE WITH MAP 



I 



54 



PRODUCE BULLSEYE RENDITION OF IMAGE 



I 



56 



TRANSFER IMAGE COLORATION TO 3- D MAP 



58 



PERFORM MAP-GUIDED THERAPEUTIC PROCEDURE 



60 



13 



BNSCOCIO: <EP. 



11B2618A2J_> 



EP1 182 619 A2 




9 



EP 1 182 619 A2 




BNSDOCID: <EP 1 1B2819A2J_> 




BNSDOCID: <EP 11B261QA2J_> 



16 




BNSDOCID: <EP 118261BA2J_> 



EP 1 182 619 A2 



o 




BNSDOCID: <EP ua2fllGA2_l_> 



18 



(19) 




Europaisches Patentamt 
European Patent Office 
Office europeen des brevets 







(12) 



(11) EP1 182 619 A3 

EUROPEAN PATENT APPLICATION 



(88) Date of publication A3: 

29.09.2004 Bulletin 2004/40 

(43) Date of publication A2: 

27.02.2002 Bulletin 2002/09 

(21) Application number 01307001.6 

(22) Date of filing: 17.08.2001 



(51) IntCI 7 : G06T 17/00 



i I* 



(84) Designated Contracting States: 


(72) Inventor: Keidar, Yaron 


AT BE CH CY DE DK ES Fl FRGB GR IE IT LI LU 


Haifa Israel 34990 (IL) 


MCNLPTSETR 




Designated Extension States: 


(74) Representative: Mercer, Christopher Paul et al 


AL LT LV MK RO SI 


Carpmaels & Ransford 




43, Bloomsbury Square 


(30) Priority: 18.08.2000 US 643666 


London WC1A2RA (GB) 


(71) Applicant: Biosense, Inc. 




New Brunswick, New Jersey 08933-7003 (US) 





(54) Method and apparatus for three-dimensional image rendering of body organs 



(57) A method for mapping a structure in a body of 
a subject includes capturing a three-dimensional (3D) 
image of the structure comprising diagnostic informa- 
tion, and generating a 3D geometrical map of the struc- 
ture using a probe inserted into the structure. The image 



FIG. 1 



is registered with the map, such that each of a plurality 
of image points in the image is identified with a corre- 
sponding map point in the map. The map Is displayed 
such that the diagnostic information associated with 
each of the image points is displayed at the correspond- 
ing map point. 



-20 



CO 

< 

CO 

CM 
00 



2j 





(17 




Printed by Jouve, 75001 PARIS (PR) 



BNSOCCID: <EP_ 



.11S2B10A3_L> 



EP 1 182 619 A3 




European Potent 
Office 



EUROPEAN SEARCH REPORT 



Application Number 

EP 01 36 7081 



DOCUMENTS CONSIDERED TO BE RELEVANT 



Category 



Citation of document with indioation, where appropriate, 
ot relevant passages 



Relevant 
to claim 



CLASSIFICATION OF THE 
APPLICATION (lm.CJ.7) 



X 
X 

D,X 



D,A 



SCHILLING ET AL: "Mapping and ablation of 
ventricular tachycardia with the aid of a 
non-contact mapping.' .system" 
HEART, 

vol. 81, no. 6, June 1999 (1999-06), pages 
570-575, XPG02284986 

* page 571, column 2, lines 26-31; 
page 572, column 1, lines 5-25 * 

* page 572, column 1, lines 21-24 * 

* figure 2 * 



1,11 



G06T17/Q0 



US 5 738 096 A (BEN-HAIM SRL0M0) 
14 April 1998 (1998-04-14) 

* column 9, line 5 - line 10 * 

J B A MAINTZ, M A VI ERGEVER: "A Survey of 

Medical Image Registration 0 

MEDICAL IMAGE ANALYSIS, 

vol. 2, no. 1, March 1998 (1998-03), pages 

1-36, XP002284987 

* page 1, column 2, lines 14-16; 
page 6, column 2, lines 17-26 * 

T L FABER ET AL: "Three-Dimensional 
Displays of Left Ventricular Epi cardial 
Surface from Standard Cardiac SPECT 
Perfusion Quantification Techniques" 
JOURNAL OF NUCLEAR MEDICINE, 
vol. 36, no. 4, April 1995 (1995-04), 
pages 697-703, XPO08031998 

* page 698 * 

EP 0 974 936 A (BI0SENSE INC) 
26 January 2000 (2000-01-26) 

* page 9, lines 39-40 * 

-/-- 



9,19 
10,20 

3,4,13, 
14 



5,15 



6-8, 
16-18 



21,24,26 



The present search report has been drawn up for a/t claims 



TECHNICAL FIELDS 
SEARCHED 0M.C1.7) 



G06T 
A61B 



8 

3 

8 



u. 

o 



Piece ol t«4rt* 

The Hague 



OeflB of conpleiten ol the search 

25 June 2004 



Examner 

Reise, F 



CATEGORY OF CITED DOCUMENTS 

X : particular relevant K taken alone 

Y : particularly relevant if combined with another 

document ol the tame category 
A : technological background 
O : non-written dhdosune 
P : intermediate document 




T : theory or principle underlying the Invention 
E : eariier patent document but published on, or 

after the tiling data 
D : document cited in the application 
L : document oited for other reasons 



ft : member of the tame patent famSy, corresponding 
document 



2 



<£P 1 18281 9A3_I_> 



EP 1 182 619 A3 




European Patent 
Office 



EUROPEAN SEARCH REPORT 



Application Number 

EP 01 30 7001 



DOCUMENTS CONSIDERED TO BE RELEVANT 



Category 



Citation oJ document with indication, where appropriate, 
of relevant passages 



Relevant 
b claim 



CLASSIFICATION OF THE 
APPLICATION (lntCI.7) 



US 5 769 843 A (ABELA GEORGE S ET AL) 
23 June 1998 (1998-06-23] 

* abstract * 

H YAMAGATA ET AL: development of a new 
display method for compound 3D ultrasound 
images: Fusion 3D images from B-mode and 
3D doppler images" 

MEDICAL REVIEW* no. 70, 1999, pages 43-46, 
XP002284988 

* abstract * 



24,25 



J L BOXERMAN, T J HOSHER. E R MCVEIGH, E 
ATALAR, J A LIMA, D A BLUEMKE: "Advanced 
MR imaging techniques for evaluation of 
the heart and great vessels" . 
RADIOGRAPHICS, 

vol. 18, no. 3, 1988, pages 543-564, 

XP002284989 

* abstract * 



GULLBERG G T ET AL: "Imaging the 

mechanical, electrical, and physiological 

properties of the heart" 

ENGINEERING IN MEDICINE AND BIOLOGY 

SOCIETY, 1998. PROCEEDINGS OF THE 20TH 

ANNUAL INTERNATIONAL CONFERENCE OF THE 

IEEE HONG KONG, CHINA 29 0CT.-1 NOV. 1998, 

PISCATAWAY, NO, USA, IEEE, US, 

29 October 1998 (1998-10-29), pages 

496-501, XPO1032O85O 

ISBN: 0-7803-5164-9 

* page 496, column 2, lines 22-30 * 

-/-- 



TECHNICAL FIELDS 
SEARCHED (ht.CJ.7) 



The present search report has been drawn up for all claims 



8 
5 

c 

5 

8 
I 

1 



Place oJ aoaich 

The Hague 



Cao Of eomplai cn a( the search 

25 June 2004 



Examiner 

Reise, F 



CATEGORY OF CITED DOCUMENTS 

X : particularly relevant if taken atone 

Y : parboil arty relevant it combined with another 

document of the come category 
A : tec hnotog tool background 
O : norrwritton d is closure 
P : intermediate document 



T : theory or principle underfyino, the invention 
6 : earlier potent document, but published on, or 

•her the tffing data 
D : document cited in the application 
L : document cited for other reasons 



& : member a! the seme patent family, corresponding 
document 



3 



BNSDOCID: <EP 1182610A3_I_> 



EP 1 182 619 A3 

v. 




European Patent 
Office 



EUROPEAN SEARCH REPORT 



Application Number 

EP Ql 30 7001 



DOCUMENTS CONSIDERED TO BE RELEVANT 



Category 



Citation of document with indication, where appropriate, 
of relevant passages 



Relevant 
to claim 



CLASSIFICATION OF THE 
APPLICATION (lntCJ.7) 



GILAROI M C ET AL: "Correlation of SPECT 
and PET cardiac images by a surface 
matching registration technique" 
COMPUT. MED. IMAGING GRAPH. (UK), 
COMPUTERIZED MEDICAL IMAGING AND GRAPHICS, 
SEPT. -OCT. 1998, ELSEVIER. UK, 
vol. 22, no. 5, 1988, pages 391-398, 
XP002284990 
ISSN: 0895-6L11 

* page 391, coluim 1, lines 1-4 * 



TECHNICAL FIELDS 
SEARCHED (lnLCI.7) 



The present search report has been drawn up for all claims 



8 
§ 



o 

Ul 



PtaOB ai mai (it 

The Hague 



>da oi compteticn ct the Bench 

25 June 2004 



Examiner 



Reise, F 



CATEGORY OF CITED DOCUMENTS 

X : particularly relevant i token obn« 

y : partiouiariy relevant if combined with another 

dooument of the some category 
A : technctog'cai background 
O : noo- written dndoium 
P : intormadiett document 



T : theory or princple underlying the invention 
E : earner patent document, but pubfiahed an, or 

after the King date 
D : dooument cited in the application 
L : dooument oitad for other racoon* 

•■«••**•** l|HliltM*f*HMH««IHII»»<HM«HIHMt*M««"*l**^«**HIMIfll«|1>tH M||M*MlflU 

& : member ot the eame patent (emit/, coneiponding 
document 



4 



BNSDOCID: <EP 1182619A3J_> 



EP 1 182 619 A3 



ANNEX TO THE EUROPEAN SEARCH REPORT 

ON EUROPEAN PATENT APPLICATION NO. £P 01 30 7001 



This annex lists the patent family members relating to the patent documents oited in the above-mentioned European search report 
The mam bo re ana as contained in the European Patent Office EDP file on 

The European Patent Office is in no way Rafale for these particulars which are merely given for the purpose of information. 

25-06-20G4 



Patent document 
cited in search report 


Publication 
date 


Patent (amity 
members) 


Pi i W Wnt iftn 
ruMiwttuwi 

date 


US 5738096 A 


14-04-1998 


US 


5546951 


A 


20-0'8-i996 




us 


5391199 


A 


21-02-1995 




a\ ■ a 

AU 


715925 


82 


10-02-2000 




a a k 

AU 


1170197 


* 

A 


01-08-1997 




AU 


710236 


B2 


16-09-1999 




AU 


1170297 


A 


01-08-1997 




AU 


712539 


B2 


11-11-1999 




AU 


1206697 


A 


01-08-1997 




AU 


724404 


r> af% 

B2 


21-09-2000 




jit 

AU 


1206797 


* 

A 


01-08-1997 




CA 


2240943 


a « 

Al 


17-07-1997 




CA 


2242353 


Al 


17-07-1997 




CA 


2242356 


Al 


17-07-1997 




CA 


2242360 


Al 


17-07-1997 




CN 


1211930 


A 


24-03-1999 




D£ 


a- /» ^ A a» *» «v 

69726599 


Dl 


15-01-2004 




DE 


69728257 


Dl 


, 29-04-2004 




EP 


1382293 


A O 

A2 


21-01-2004 




EP 


1428472 


AO 

A2 


16-06-2004 




EP 


0888082 


A 1 

Al 


07-01-1999 




EP 


ftft >l dOEfl 

0944350 


A 1 

Al 


29-09-1999 




EP 


0888150 


A 1 

Al 


07-01-1999 




EP 


0910429 


Al 


28-04-1999 




WO 


mo /inoi 

9724983 


AO 

AZ 


17-07-1997 




WO 


9724981 


A2 


17-07-1997 




wo 


9725101 


A2 


17-07-1997 




wo 


9725098 


Al 


17-07-1997 




wo 


9937208 


Al 


29-07-1999 




IL 


125136 


A 


31-0A-ZUW3 




IL 


125259 


A 


01-12-2002 




IL 


125260 


A 


24-06-2003 




US 


6066094 


A 


23-05-2000 




US 


6317631 


Bl 


13-11-2001 




US 


6285898 


Bl 


04-09-2001 




US 


2003139668 


Al 


24-07-2003 




US 


6498944 


Bl 


24-12-2002 




US 


2003191383 


Al 


09-10-2003 




US 


6363279 


Bl 


26-03-2002 




US 


6330476 


Bl 


11-12-2001 




US 


2002045809 


Al 


18-04-2002 




us 


2002045812 


Al 


18-04-2002 




AT 


188108 


T 


15-01-2000 




AT 


253864 


T 


15-11-2003 




AU • 


1693095 


A 


14-03-1996 




CA 


2197986 


Al 


29-02-1996 




CN 


1168625 


A 


24-12-1997 



For more details about this annex : see Offioiol Journal of the European Patent Offioe, No. 1 2/B2 



5 



1182619A3J_> 



EP 1 182 619 A3 



ANNEX TO THE EUROPEAN SEARCH REPORT 
ON EUROPEAN PATENT APPLICATION NO. 



EP 01 30 7001 



This annex lists the paten! family member* relating to the patent doeumenta cited In the above-mentioned European search report. 
The members are as contained in the European Patent Office EDP fib on 

The European Patent Office is in no way liable for these particulars whioh are merely given tor the purpose of information. 

25-06-2004 



Patent document 
cited in search report 



Publication 
date 



Patent family 
member^) 



US 5738096 



0E 
DE 



69514238 Dl 
69514238 T2 



EP 0974936 



26-01-2000 



US 6226542 Bl 

CA 2278380 Al 

EP 0974936 A2 

US 6301496 Bl 

US 2001009974 Al 



US 5769843 



23-06-1998 



CA 2247505 Al 

EP 0901399 Al 

JP 2001509036 T 

W0 9729803 Al 



Publication 
date 



03-02-2000 
11-05-2000 



01-05-2001 
24-01-2000 
26-01-2000 
09-10-2001 
26-07-2001 



21-08-1997 
17-03-1999 
10-07-2001 
21-08-1997 



& For more details about this annex : see OffioialJoumal of the European Patent Office, No. 12/82 



BNSDOCID: <EP 1 1 B261 9A3_I_> 



1 



This Page is Inserted by IFW Indexing and Scanning 
Operations and is not part of the Official Record 

BEST AVAILABLE IMAGES 

Defective images within this document are accurate representations of the original 
documents submitted by the applicant. 

Defects in the images include but are not limited to the items checked: 

□ BLACK BORDERS 

* 

□ IMAGE CUT OFF AT TOP, BOTTOM OR SIDES 

* 

□ FADED TEXT OR DRAWING 

□ BLURRED OR ILLEGIBLE TEXT OR DRAWING 

□ SKEWED/SLANTED IMAGES 

□ COLOR OR BLACK AND WHITE PHOTOGRAPHS 

□ GRAY SCALE DOCUMENTS 

□ LINES OR MARKS ON ORIGINAL DOCUMENT 

□ REFERENCE(S) OR EXHIBIT(S) SUBMITTED ARE POOR QUALITY 

□ OTHER: 

IMAGES ARE BEST AVAILABLE COPY. 
As rescanning these documents will not correct the image 
problems checked, please do not report these problems to 
the IFW Image Problem Mailbox. 



